High dynamic range processing

ABSTRACT

An apparatus for HDR image processing is provided. The apparatus determines an imaging sensitivity value. The apparatus then compares subsets of imaging information with the determined imaging sensitivity value and applies a gamma correction to each subset of imaging information using a gamma low contrast curve or a gamma high contrast curve based on the comparison to obtain the gamma corrected subset of imaging information.

CROSS REFERENCE TO RELATED APPLICATIONS

The current application claims priority to U.S. Patent ProvisionalApplication No. 62/384,606, filed Sep. 7, 2016, the entire contents ofwhich are hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates generally to image processing, and moreparticularly, to high dynamic range processing.

BACKGROUND

Display devices, including cameras, computer monitors, mobiletelephones, televisions, and personal devices, may include manyindividual pixels for displaying images and videos. Each pixel maygenerate a range of colors at a particular point in a display. Together,the pixels may be used to form an image or video by displaying differentcolors at different intensities or luminance on a display screen.

Many devices display images with a standard dynamic range (SDR). SDRdescribes the dynamic range of images and videos adjusted using aconventional gamma curve. The conventional gamma curve was based oncertain limitations of the cathode ray tube (CRT) and allowed for amaximum luminance 100 candela per square meter (cd/m²). Recently, imagecapture devices, such as cameras, have begun to support high dynamicrange (HDR) imaging, which is a technique that enables a greater dynamicrange of luminosity as compared to SDR. HDR may provide higher contrastand brightness characteristics that simulate the human vision. Advancesin ways to capture an image must be accompanied by the development andimprovement of the methods for displaying the image capture results.

SUMMARY

The following presents a simplified summary of one or more aspects inorder to provide a basic understanding of such aspects. This summary isnot an extensive overview of all contemplated aspects, and is intendedto neither identify key or critical elements of all aspects nordelineate the scope of any or all aspects. Its sole purpose is topresent some concepts of one or more aspects in a simplified form as aprelude to the more detailed description that is presented later.

Because of the large dynamic range of HDR images, image processing mayneed to be adjusted in order to enable users to experience the benefitof HDR images. In an aspect of the disclosure, a method, acomputer-readable medium, and an apparatus are provided. The apparatusmay be configured to determine an imaging sensitivity value. Theapparatus may be configured to compare subsets of imaging informationwith the determined imaging sensitivity value. The apparatus may beconfigured to apply a gamma correction to each subset of imaginginformation using a gamma low contrast curve or a gamma high contrastcurve based on the comparison to obtain the gamma corrected subset ofimaging information.

Therefore, according to an exemplary embodiment, an image processingsystem is provided for adjusting luminosity of a high-dynamic range(HDR) image based on imaging sensitivity of a device configured todisplay the HDR image. In this embodiment, the image processing systemincludes electronic memory configured to store at least one gamma lowcontrast curve and at least one a gamma high contrast curve; an imagingapparatus for generating image data of a captured HDR image; an imagingsensitivity determiner configured to determine an imaging sensitivityvalue of a display device based on a input value of the display devicethat provides a predetermined output luminance at display device; animage information comparator configured to compare the determinedimaging sensitivity value of the display device with a luminance valueof the generated image data; a gamma curve selector configured to selectthe at least one gamma low contrast curve when the luminance value isless than the determined imaging sensitivity value of the display deviceand select the at least one gamma high contrast curve when the luminancevalue is greater than the determined imaging sensitivity value of thedisplay device; an image corrector configured to correct the image dataof the captured HDR image by applying the selected at least one gammahigh contrast curve or the selected at least one gamma low contrastcurve to the generated image data to adjust the luminosity of the HDRimage; and an image display configured display the adjusted HDR image onthe display device.

In another exemplary aspect, an image processing system is provided foradjusting luminosity of a high-dynamic range (HDR) image based onimaging sensitivity of a device configured to display the HDR image. Inthis aspect, the image processing system includes electronic memory forstoring a plurality of gamma contrast curves; an imaging apparatus forgenerating image data of a captured HDR image; an imaging sensitivitydeterminer configured to determine an imaging sensitivity value of adisplay device; an image information comparator configured to comparethe determined imaging sensitivity value of the display device with aluminance value of the generated image data to generate a gamma curveselection determination; a gamma curve selector configured to select atleast one gamma contrast curve from the plurality of gamma contrastcurves based on the generated gamma curve selection determination; andan image corrector configured to correct the image data of the capturedHDR image by applying the selected at least one gamma contrast curve tothe generated image data to adjust the luminosity of the HDR image.

In yet another exemplary aspect, an image processor is provided forsetting a luminosity of an image based on an imaging sensitivity of adisplay device. In this aspect, the image processor includes a luminanceadjustment selector configured to select at least one luminanceadjustment modifier by comparing a luminance of a captured image with animaging sensitivity of a display device to determine an optimalluminance level for the captured image when displayed on the displaydevice; and an image enhancer configured to apply the at least oneluminance adjustment modifier to the captured image to generate anenhanced image configured to be displayed on the display device with theoptimal luminance level for the display device.

To the accomplishment of the foregoing and related ends, the one or moreaspects comprise the features hereinafter fully described andparticularly pointed out in the claims. The following description andthe annexed drawings set forth in detail certain illustrative featuresof the one or more aspects. These features are indicative, however, ofbut a few of the various ways in which the principles of various aspectsmay be employed, and this description is intended to include all suchaspects and their equivalents.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an image capture and display network according toan exemplary embodiment.

FIG. 2 is a diagram of a first set of OOTF contrast curves according toan exemplary embodiment.

FIG. 3 is a diagram of a second set of OOTF contrast curves according toan exemplary embodiment.

FIG. 4 is a diagram of a first method of high dynamic range processingaccording to an exemplary embodiment.

FIG. 5 is a diagram of a second method of high dynamic range processingwith noise consideration according to an exemplary embodiment.

FIG. 6 is a diagram of a method for converting linear values intologarithmic values according to an exemplary embodiment.

FIG. 7 is a diagram illustrating base 2 exponent values according to anexemplary embodiment.

FIG. 8 is a diagram of a method for converting logarithmic values intolinear values according to an exemplary embodiment.

FIG. 9 illustrates a Matlab implementation of the HDR processingtechniques according to an exemplary embodiment.

FIG. 10 is a flowchart of a method of HDR processing according to anexemplary embodiment.

FIG. 11 is a conceptual data flow diagram illustrating the data flowbetween different components of the disclosed system according to anexemplary embodiment.

FIG. 12 is a diagram illustrating an example of a hardwareimplementation for an apparatus employing a processing system accordingto an exemplary embodiment.

DETAILED DESCRIPTION

The detailed description set forth below in connection with the appendeddrawings is intended as a description of various configurations and isnot intended to represent the only configurations in which the conceptsdescribed herein may be practiced. The detailed description includesspecific details for the purpose of providing a thorough understandingof various concepts. However, it will be apparent to those skilled inthe art that these concepts may be practiced without these specificdetails. In some instances, well known structures and components areshown in block diagram form in order to avoid obscuring such concepts.

Several aspects of display systems will now be presented with referenceto various apparatus and methods. These apparatus and methods will bedescribed in the following detailed description and illustrated in theaccompanying drawings by various blocks, components, circuits,processes, algorithms, etc. (collectively referred to as “elements”).These elements may be implemented using electronic hardware, computersoftware, or any combination thereof. Whether such elements areimplemented as hardware or software depends upon the particularapplication and design constraints imposed on the overall system.

By way of example, an element, or any portion of an element, or anycombination of elements may be implemented as a “processing system” thatincludes one or more processors. Examples of processors includemicroprocessors, microcontrollers, graphics processing units (GPUs),central processing units (CPUs), application processors, digital signalprocessors (DSPs), reduced instruction set computing (RISC) processors,systems on a chip (SoC), baseband processors, field programmable gatearrays (FPGAs), programmable logic devices (PLDs), state machines, gatedlogic, discrete hardware circuits, and other suitable hardwareconfigured to perform the various functionality described throughoutthis disclosure. One or more processors in the processing system mayexecute software. Software shall be construed broadly to meaninstructions, instruction sets, code, code segments, program code,programs, subprograms, software components, applications, softwareapplications, software packages, routines, subroutines, objects,executables, threads of execution, procedures, functions, etc., whetherreferred to as software, firmware, middleware, microcode, hardwaredescription language, or otherwise.

Accordingly, in one or more example embodiments, the functions describedmay be implemented in hardware, software, or any combination thereof. Ifimplemented in software, the functions may be stored on or encoded asone or more instructions or code on a computer-readable medium.Computer-readable media includes computer storage media. Storage mediamay be any available media that can be accessed by a computer. By way ofexample, and not limitation, such computer-readable media can comprise arandom-access memory (RAM), a read-only memory (ROM), an electricallyerasable programmable ROM (EEPROM), optical disk storage, magnetic diskstorage, other magnetic storage devices, combinations of theaforementioned types of computer-readable media, or any other mediumthat can be used to store computer executable code in the form ofinstructions or data structures that can be accessed by a computer.

FIG. 1 is a diagram of an image capture and display network 100.Referring to FIG. 1, a camera 102 (or any other type of image or videocapture device) may generate image data 110 (or video data) based on anenvironment. To capture the image data 110, the camera 102 may utilizean opto-electronic transfer function (OETF) to convert the optical datainto electrical signals such as the image data 110. The OETF may be acamera transfer function. The image data 110 may be displayed from thecamera 102 itself or may be transmitted via a distribution network 112to a personal computer 104, a user equipment 106, and/or a television108, among other display devices.

To display the image data 110, the camera 102 or the other displaydevices may apply an electrical-optical transfer function (EOTF) thatconverts the electrical signals of the image data 110 into opticalsignals for display. In an aspect, the camera 102 or the other displaydevices may also perform an electrical-electrical transfer function(EETF) before the EOTF. The EETF may map the incoming image data 110received from the camera 102 into a different set of electronic signalsformatted according to the display capabilities of the camera 102 or theother display devices.

A fourth transfer function, known as an opto-optical transfer function(OOTF), may also be utilized. The OOTF function may be used to modifyoptical signals from one form to another (e.g., modifying the contrastproperties of an image). Although this disclosure may refer to images,one of ordinary skill in the art would understand that any methods,techniques, and protocols applied to an image may be similarly appliedto videos.

For SDR, transfer functions may be standardized. For example, the SDROETF is standardized according to ITU-R BT.709 (camera gamma), and theSDR EOTF is standardized under ITU-R BT.1886 (CRT response). Aside fromOETF/EOTF, a broadcast camera may also have artistic adjustments on theOETF such as variable gamma and contrast curves. Displays may also havebrightness and contrast adjustments.

For HDR, OETF and EOTF transfer functions have been defined. Thoseinclude the SMPTE2084 and the Hybrid Log Gamma (HLG) transfer function.SMPTE2084 and HLG transfer functions may be used for OETF or EOTF, whichare inverses of one another. In HDR OETF, the camera sensitivity mayneed to be mapped on a point in the HDR chain. For example, if thecamera is set at F12 with a 2000 LUX and 90% reflection, SDR videooutput would be at 100% or 100 cd/m².

In another aspect, SMPTE 2084 defines the EOTF with a normalized inputof [0,1]. In this aspect, a value of 0.5 at the input provides 100 cd/m²of display luminance. Experimentally, this has been found to be a goodoperational point for 100% camera sensitivity. For HLG, an input of 0.5also results in 100% camera sensitivity.

Because of the wide dynamic range in HDR, additional rendering of theluminosity may be needed for a display to provide the full rangecontrasts. In an aspect, for HDR, the lowlight portions of an imageshould appear similar to the lowlight portions of the image in SDR. Toproperly display the high contrast, an OOTF may be used to keep theresponse for low light similar to that as in SDR. For example, one maymeasure the patches of a gamma chart on a reference display in SDR mode.Users may be allowed to adjust the camera OOTF in HDR mode such that theluminosity levels at the monitor output (in cd/m²) for example, are thesame.

FIG. 2 is a diagram 200 of a first set of OOTF contrast curves. Thethicker curve line represents the calculated SDR OOTF curve. The SDROOTF curve is based on a gamma of 1.2 to 1.4 based on BT1886 (Rec709).The thinner darker curves represent the HDR paddock curves for LDXcameras. The X and Y axis represent units of luminance such as cd/m².

FIG. 3 is a diagram 300 of a second set of OOTF contrast curves.Referring to FIG. 3, a display device (e.g., the camera 102) may have animaging sensitivity value. The imaging sensitivity value may be 100cd/m². The display device may provide different gamma contrast curvesdepending on whether the imaging information for which gamma correctionis to be applied is associated with a luminance value that is greater orless than the imaging sensitivity value. If the luminance value is lessthan the imaging sensitivity value, then a gamma low contrast curve maybe applied. In an aspect, the gamma low contrast curve may have a sloperanging from [1.0, 3.0]. When the imaging information has a luminositythat is less than the imaging sensitivity value, a gamma value ofgreater than 1 attenuates the luminosity, while a gamma value of lessthan 1 amplifies the luminosity. Preferably, the luminosity isattenuated or amplified to match and optimal luminance level for thecaptured image when it is displayed on the display device.

According to a first exemplary aspect, the imaging sensitivity value canbe a defined breakpoint, such as a predetermined luminance value of thedisplay device, for example. However, it should be appreciated that theimaging sensitivity value is not necessarily limited to a predeterminedbreakpoint. For example, in an alternative aspect, the imagingsensitivity value can be a defined range of luminance values for thedisplay device. In this aspect, the one or more luminance adjustmentvalues (e.g., one or more gamma correction values or curves) can beselected based on the comparison between the luminance value of thecaptured image and the range of imaging sensitivity values for thedisplay device. Moreover, the imaging sensitivity value can bedetermined by an imaging sensitivity determiner, for example, a moduleof the processing as will be described below.

In another aspect, if the imaging information is associated with aluminance value that exceeds the imaging sensitivity value, then a gammahigh contrast curve may be applied. The gamma high contrast curve mayhave a different slope than the gamma low contrast curve. In an aspect,the gamma high contrast curve may have a slope ranging from [0.8, 1.2].When the imaging information has a luminosity that is greater than theimaging sensitivity value, a gamma value of greater than 1 may amplifythe luminosity, while a gamma value of less than 1 attenuates theluminosity.

In an aspect, a gamma high contrast curve may be used to highlightcompression. The value of the gamma high contrast curve may be used inmetadata to indicate the operating maximum luminance. Other metadataparameters may also be provided.

FIG. 4 is a diagram 400 of a first method of high dynamic rangeprocessing. Referring to FIG. 4, a display device (e.g., a camera or amonitor) may receive imaging information (e.g., raw image data or rawvideo data). The imaging information may be in the linear domain. Toreduce multiplication, quotient, power, and root operations tomultiplication, addition, and subtraction operations, the imaginginformation may be converted from the linear domain to the logarithmicdomain. In an aspect, real-time processing by an FPGA, for example, maybe quicker after converting the imaging information into the logarithmicdomain. A first processor 402 may convert the linear imaging informationinto logarithmic imaging information—although such conversion is notrequired for HDR processing. The display device may determine an imagingsensitivity value associated with the display device. In an aspect, theimaging sensitivity value may be an input value that results in anoutput luminance of 100 cd/m² at the display device (or the otherdisplay device). In another aspect, the imaging sensitivity value may bepreconfigured within the display device.

After converting the imaging information into the logarithmic domain, asubset of the imaging information (e.g., luminance informationcorresponding to one or more pixels) may be compared with the imagingsensitivity value by a comparator 404. In particular, the comparator 404of the display device may determine whether the subset of imaginginformation is associated with a luminance that is greater than theimaging sensitivity value. The result of the comparison may be used todetermine whether to apply a gamma high contrast curve or a gamma lowcontrast curve on the subset of imaging information. The display devicemay subtract 406 from the subset of imaging information a normalizationvalue (which may be equal to the imaging sensitivity value). Thedifference may be subjected to a gamma high contrast curve and/or agamma low contrast curve. In an aspect, if the subset of imaginginformation is associated with a luminance that is greater than theimaging sensitivity value, then the normalized subset of imaginginformation subjected to a gamma high contrast curve is selected via aswitch 408. In another aspect, if the subset of imaging information isassociated with a luminance that is less than the imaging sensitivityvalue, then the normalized subset of imaging information subjected to agamma low contrast curve is selected via the switch 408. The result ofthe selection from the switch 408, which may be referred to as anintermediate subset of imaging information, may be added to thenormalization value (e.g., with an adder 410) to generate a gammacorrected subset of imaging information. Thus, it should be appreciatedthat the switch 408 can be controlled by the processing system of thedisclosed system and is configured to selected the luminance adjustmentmodifier (e.g., the particular gamma adjustment value and/or gammacontrast curve) by comparing a luminance of the captured image with theimaging sensitivity of the display device to determine an optimalluminance level for the captured image when the image is ultimatelydisplayed on the display device.

In an exemplary aspect, the display device may convert the gammacorrected subset of imaging information from the logarithmic domain backto the linear domain via a second processor 412 (which may be the sameas the first processor 402). The display device may include an imagecorrector or image enhancer that can generate a gamma corrected image byrepeatedly performing the aforementioned operations on all of thesubsets of imaging information until the operation is performed on theentire set of imaging information. In an aspect, the gamma correctedimage may have been subjected to both a gamma low contrast curve and agamma high contrast curve and/or multiple variations of the gamma lowcontrast curves and gamma high contrast curves. Although not shown inFIG. 4, the system 400 includes an image display for displaying thecorrected/adjusted image on a display device, i.e., displaying the gammacorrected image.

FIG. 5 is a diagram 500 of a second method of high dynamic rangeprocessing with noise consideration. In some instances, theamplification introduced by the gamma contrast curves may result inexcessive noise to the imaging information. As such, any amplificationmay be limited by a maximum differential gain. Referring to FIG. 5, adisplay device (e.g., a camera or a monitor) may receive imaginginformation (e.g., raw image data or raw video data). The imaginginformation may be in the linear domain. To reduce multiplication,quotient, power, and root operations to multiplication, addition, andsubtraction operations, the imaging information may be converted fromthe linear domain to the logarithmic domain. In an aspect, real-timeprocessing by an FPGA, for example, may be quicker after converting theimaging information into the logarithmic domain. A first processor 502may convert the linear imaging information into logarithmic imaginginformation—although such conversion is not required for HDR processing.The display device may determine an imaging sensitivity value associatedwith the display device. In an aspect, the imaging sensitivity value maybe an input value that results in an output luminance of 100 cd/m² atthe camera (or the other display device). In another aspect, the imagingsensitivity value may be preconfigured within the display device.

After converting the imaging information into the logarithmic domain, asubset of the imaging information (e.g., luminance informationcorresponding to one or more pixels) may be compared with the imagingsensitivity value by a comparator 504. In particular, the comparator 504of the display device may determine whether the subset of imaginginformation is associated with a luminance that is greater than theimaging sensitivity value. The result of the comparison may be used todetermine whether to apply a gamma high contrast curve or a gamma lowcontrast curve on the subset of imaging information. The display devicemay subtract 506 from the subset of imaging information a normalizationvalue. The difference may be subjected to a gamma high contrast curveand/or a gamma low contrast curve. In an aspect, if the subset ofimaging information is associated with a luminance that is greater thanthe imaging sensitivity value, then the normalized subset of imaginginformation subjected to a gamma high contrast curve is selected via aswitch 508. In another aspect, if the subset of imaging information isassociated with a luminance that is less than the imaging sensitivityvalue, then the normalized subset of imaging information subjected to agamma low contrast curve is selected via the switch 508. The result ofthe selection from the switch 508, which may be referred to as anintermediate subset of imaging information, may be added to thenormalization value (e.g., with an adder 510) to generate a secondintermediate subset of imaging information. In an aspect, a second adder514 may be used to add the subset of imaging information with a maximumdifferential gain value to generate a maximum total value. A secondcomparator 512 may compare the maximum total value with the secondintermediate subset of imaging information and select the smaller of thetwo inputs to use as the gamma corrected subset of imaging information.By utilizing the smaller of the two inputs, the display device may usethe maximum differential gain value to limit the amount of noise in thegamma corrected subset of imaging information. Subsequently, the displaydevice may convert the gamma corrected subset of imaging informationfrom the logarithmic domain back to the linear domain via a secondprocessor 516 (which may be the same as the first processor 502). Thedisplay device may generate a gamma corrected image by repeatedlyperforming the aforementioned operations on all of the subsets ofimaging information until the operation is performed on the entire setof imaging information. In an aspect, the gamma corrected image may havebeen subjected to both a gamma low contrast curve and a gamma highcontrast curve and/or multiple variations of the gamma low contrastcurves and gamma high contrast curves.

In an aspect, the HDR processing methods discussed with respect to FIGS.4 and 5 may represent a relative system. Imaging sensitivity may benormalized to a certain output value, and the display device maygenerate values relative to the normalized sensitivity value.Subsequently, gamma low and gamma high contrast curves may be used tomodify the generated values.

Different methods or techniques may be used to convert the imaginginformation from the linear domain to the logarithmic domain and viceversa. FIGS. 6 and 8 illustrate methods for linear to logarithmicconversions and vice versa. Other methods or techniques may also beused.

FIG. 6 is a diagram 600 of a method for converting linear values intologarithmic values. The operations in FIG. 6 may be performed by one ormore processors. For example, referring to FIG. 6, the processor mayreceive a linear value (Vlin). The processor may determine if the linearvalue is zero. If the linear value is zero, then a correspondinglogarithmic value is provided by a handle zero component. If the linearvalue is non-zero, the processor may calculate the absolute value of thelinear value and extract the sign of the linear value. The processor maydetermine an exponent associated with the linear value and calculate theratio of the linear value divided by the exponent. The results of theratio may be inputted into a most significant bit (MSB) component and aleast significant bit (LSB) component. The output of the MSB componentmay be provided to a lookup table (LUT) and a delta LUT. The output ofthe LSB component may be multiplied with the output from the delta LUTcomponent. The product may be added to the output of the LUT component,and the sum may be added to the exponent to generate a second sum. Thesecond sum may be the logarithmic value.

The design approach is to first determine the exponent for a base 2 logvalue by comparing 2¹, 2², 2³, . . . 2^(n). When the exponent is found,the interpolation within each octave is identical. The mantissa in theoctave is found by dividing the linear value with the determinedexponent. This is a shift operation because it is a base 2 operation.After division with the exponent, the result is always a mantissa valuebetween 1 and 2 (with 2 being the base). The interpolation value may bea value between 1 and 2, and the interpolation value may be provided tothe LUT. To achieve higher precision, the LUT may use linearinterpolation between the point in the LUT with a LUT holding deltavalues to the next point in the LUT. FIG. 7 is a diagram 700illustrating base 2 exponent values.

The absolute precision of the interpolation with the LUT for each octaveis degrading, but because the function is used for images or video, therelative precision is more important. In an aspect, a 1 percent limitmay be used to be non-visible.

FIG. 8 is a diagram 800 of a method for converting logarithmic valuesinto linear values. The operations in FIG. 8 may be performed by one ormore processors. Referring to FIG. 8, a logarithmic to linear conversionmay work similar as the linear to logarithmic conversion. Thelogarithmic value (Vlog) may be an unsigned 16 bit value of which 5 bitsare the exponent and 11 bits are the mantissa. A 5 bit exponent valuemay hold 2⁵ or 32 bits, and the mantissa may hold 11 bits for precisionwithin the octave. As shown in FIG. 8, the exponent are the mantissa areseparated by extracting the 5 MSBs and the 11 LSBs of the logarithmicvalue. The mantissa may be used as an input for a LUT, which may hold npoints within the octave. The amount of MSBs to extract from themantissa may be

$\frac{\log\mspace{11mu} n}{\log\mspace{11mu} 2}.$For 64 points in the LUT, 5 bits may be extracted. The remaining bitsfrom the mantissa (e.g., 6 in this example) may be linear interpolationsbetween the two LUT points. The interpolation uses the value stored inthe delta LUT, which is the delta to the next point in the LUT. Havingthe delta LUT may increase processing speed. Subsequently, the results,as shown in FIG. 8, are used to generate the linear value (Vlin).

FIG. 9 illustrates a Matlab implementation 900 of the HDR processingtechniques. Referring to FIG. 9, Initialization_VarGamma.m may be usedto calculate the values needed in the LUT, tb_VarGamma.slx may be a testbench for the linear to logarithmic conversion, Var Gamma, andlogarithmic to linear conversion, and GV_LIB_LinLogLin.slx may belibrary functions for Lin2 Log and Log 2Lin. The HDLCoder may be used togenerate code from block ‘VAG_VarGamma.’ The Matlab implementation mayalso use the following input/output port list shown in Table 1.

TABLE 1 Input-Output Port List Pin name Format Type Description R, G, B,C ufix(1, 15, 0) Input Video inputs - 4 channels to enable 4k RR, GG,BB, CC ufix(1, 15, 0) Output Video outputs Headroom_log2 ufix(1, 16, 8)Control input Normalization value for the gamma high and gamma lowtransition point Gamma_low ufix(0, 8, 6) Control input Gamma low valueGamma_high ufix(0, 8, 6) Control input Gamma high value Bypass BooleanControl input Bypass function (delay is not compensated due to resourcereduction) FSi Boolean Timing input FrameSync In FSo Boolean TimingFrameSync Out output

The initialization script may hold parameters for input and outputformats and the LUT sizes that are fully parametrized forMatlab/Simulink. During code generation, these should be set to thedesired values because the HDLcoder may not be generating generics.

FIG. 10 is a flowchart 1000 of a method of HDR processing. The methodmay be performed by an apparatus (e.g., a processor, a display device,the apparatus 1102/1102′). At 1002, the apparatus may receive linearimaging information. The imaging information may indicate luminancevalues for one or more pixels.

At 1004, the apparatus may convert the linear imaging information intologarithmic imaging information. For example, the luminance values maybe converted from a linear domain to a logarithmic domain.

At 1006, the apparatus may determine an imaging sensitivity value. Forexample, the imaging sensitivity value may be preconfigured. Theapparatus may determine the imaging sensitivity value by looking up thevalue in a memory location and extracting the value. For example, theimaging sensitivity value may be 100 cd/m².

At 1008, the apparatus may compare subsets of imaging information withthe determined imaging sensitivity value. The imaging information may bethe converted logarithmic imaging information. Subsets of imaginginformation may correspond to luminance values for one or more pixels.The apparatus may determine whether the imaging information indicates aluminance value greater than or equal to the imaging sensitivity value.

At 1010, the apparatus may apply a gamma correction to each subset ofimaging information using a gamma low contrast curve or a gamma highcontrast curve based on the comparison in order to obtain a gammacorrected subset of imaging information.

FIG. 11 is a conceptual data flow diagram 1100 illustrating the dataflow between different components in an exemplary apparatus 1102. Theapparatus may be a processor or a display device. The apparatus includesa Lin2 Log component 1104, a comparator component 1106, a gammacomponent 1108, and a Log 2Lin component 1110. The gamma component 1108may be configured to determine an imaging sensitivity value. Thecomparator component 1106 may be configured to compare subsets ofimaging information with the determined imaging sensitivity value. Thegamma component 1108 may be configured to apply a gamma correction toeach subset of imaging information using a gamma low contrast curve or agamma high contrast curve based on the comparison to obtain the gammacorrected subset of imaging information. For example, the gammacomponent 1108 can include computer/electronic memory for storing one ormore gamma low contrast curves and a gamma high contrast curvesaccording to the exemplary embodiment.

Moreover, in an exemplary aspect, the imaging sensitivity value may bean input value that results in an output luminance of 100 cd/m² at theapparatus. In another aspect, the imaging sensitivity value may bepreconfigured within the apparatus. In another configuration, theapparatus may include an interface (e.g., a bus interface). Theinterface may be configured to receive linear imaging information. Inthis configuration, the Lin2 Log component 1104 may be configured toconvert the linear imaging information into logarithmic imaginginformation. The logarithmic imaging information may be the imaginginformation. In another configuration, the comparator component 1106 maybe configured to compare the subsets of imaging information bydetermining whether each subset of imaging information corresponds to afirst luminance that is greater than a second luminance associated withthe imaging sensitivity value. In another configuration, the gammacomponent 1108 may be configured to apply the gamma correction bynormalizing the subset of imaging information based on a normalizationvalue, by selecting the gamma low contrast curve if the subset ofimaging information corresponds to a first luminance that is less than asecond luminance associated with the imaging sensitivity value, and byselecting the gamma high contrast curve if the subset of imaginginformation corresponds to a third luminance that is greater than thesecond luminance associated with the imaging sensitivity value. In thisconfiguration, the selected gamma low contrast curve or the selectedgamma high contrast curve may be applied to the normalized subset ofimaging information to obtain an intermediate subset of imaginginformation. In another aspect, the normalization value may be equal tothe imaging sensitivity value. In another configuration, the gammacomponent 1108 may be configured to apply the gamma correction by addingthe normalization value to the intermediate subset of imaginginformation to obtain the gamma corrected subset of imaging information.In another configuration, the Log 2Lin component 1110 may be configuredto convert the gamma corrected subset of imaging information from alogarithmic format to a linear format. In another configuration, thegamma component 1108 may be configured to apply the gamma correction bycomputing a maximum total value by adding the subset of imaginginformation with a maximum differential gain value and by computing asecond intermediate subset of imaging information by adding thenormalizing value to the intermediate subset of imaging information. Thegamma corrected subset of imaging information may be a smaller of themaximum total value or the second intermediate subset of imaginginformation. In another configuration, the gamma component 1108 may beconfigured to generate a gamma corrected image by accumulating gammacorrected subsets of imaging information. The gamma corrected image mayhave been subjected to both the gamma low contrast curve and the gammahigh contrast curve.

The apparatus may include additional components that perform each of theblocks of the algorithm in the aforementioned flowcharts of FIG. 10. Assuch, each block in the aforementioned flowcharts of FIG. 10 may beperformed by a component and the apparatus may include one or more ofthose components. The components may be one or more hardware componentsspecifically configured to carry out the stated processes/algorithm,implemented by a processor configured to perform the statedprocesses/algorithm, stored within a computer-readable medium forimplementation by a processor, or some combination thereof.

FIG. 12 is a diagram 1200 illustrating an example of a hardwareimplementation for an apparatus 1102′ employing a processing system1214. The processing system 1214 may be implemented with a busarchitecture, represented generally by the bus 1224. The bus 1224 mayinclude any number of interconnecting buses and bridges depending on thespecific application of the processing system 1214 and the overalldesign constraints. The bus 1224 links together various circuitsincluding one or more processors and/or hardware components, representedby the processor 1204, the components 1104, 1106, 1108, 1110 and thecomputer-readable medium/memory 1206. The bus 1224 may also link variousother circuits such as timing sources, peripherals, voltage regulators,and power management circuits, which are well known in the art, andtherefore, will not be described any further.

The processing system 1214 may be coupled to a transceiver 1210. Thetransceiver 1210 is coupled to one or more antennas 1220. Thetransceiver 1210 is configured to communicate with various otherapparatus over a transmission medium. The transceiver 1210 receives asignal from the one or more antennas 1220, extracts information from thereceived signal, and provides the extracted information to theprocessing system 1214 (e.g., the Lin2 Log component 1104). In addition,the transceiver 1210 receives information from the processing system1214, and based on the received information, generates a signal to beapplied to the one or more antennas 1220. The processing system 1214includes a processor 1204 coupled to a computer-readable medium/memory1206. The processor 1204 is responsible for general processing,including the execution of software stored on the computer-readablemedium/memory 1206. The software, when executed by the processor 1204,causes the processing system 1214 to perform the various functionsdescribed supra for any particular apparatus. The computer-readablemedium/memory 1206 may also be used for storing data that is manipulatedby the processor 1204 when executing software. The processing system1214 further includes at least one of the components 1104, 1106, 1108,1110. The components may be software components running in the processor1204, resident/stored in the computer readable medium/memory 1206, oneor more hardware components coupled to the processor 1204, or somecombination thereof. The processing system 1214 may be a component of adisplay device and may include memory and one or more processors coupledto the memory.

In one configuration, the apparatus 1102/1102′ for image processing isconfigured to determine an imaging sensitivity value (i.e., an imagingsensitivity determiner). The apparatus may include image informationcomparator for comparing subsets of imaging information with thedetermined imaging sensitivity value. The apparatus may include an imagecorrector or image enhancer for applying a gamma correction to eachsubset of imaging information using a gamma low contrast curve or agamma high contrast curve based on the comparison to obtain the gammacorrected subset of imaging information. In an aspect, the imagingsensitivity value may be an input value that results in an outputluminance of 100 cd/m² at the apparatus. In another aspect, the imagingsensitivity value may be preconfigured within the apparatus.

In another configuration, the apparatus may include a receiver forreceiving linear imaging information. In this configuration, theapparatus may include a converter for converting the linear imaginginformation into logarithmic imaging information. The logarithmicimaging information may be the imaging information. In anotherconfiguration, the image information comparator for comparing thesubsets of imaging information may be configured to determine whethereach subset of imaging information corresponds to a first luminance thatis greater than a second luminance associated with the imagingsensitivity value. In another configuration, the imagecorrector/enhancer for applying the gamma correction may be configuredto normalize the subset of imaging information based on a normalizationvalue, to select the gamma low contrast curve if the subset of imaginginformation corresponds to a first luminance that is less than a secondluminance associated with the imaging sensitivity value, and to selectthe gamma high contrast curve if the subset of imaging informationcorresponds to a third luminance that is greater than the secondluminance associated with the imaging sensitivity value. In thisconfiguration, the selected gamma low contrast curve or the selectedgamma high contrast curve may be applied to the normalized subset ofimaging information to obtain an intermediate subset of imaginginformation. In another aspect, the normalization value may be equal tothe imaging sensitivity value. In another configuration, the imagecorrector/enhancer for applying the gamma correction may be configuredto add the normalization value to the intermediate subset of imaginginformation to obtain the gamma corrected subset of imaging information.In another configuration, the apparatus may include a converter forconverting the gamma corrected subset of imaging information from alogarithmic format to a linear format. In another configuration, theimage corrector/enhancer for applying the gamma correction may beconfigured to compute a maximum total value by adding the subset ofimaging information with a maximum differential gain value and tocompute a second intermediate subset of imaging information by addingthe normalizing value to the intermediate subset of imaging information.The gamma corrected subset of imaging information may be a smaller ofthe maximum total value or the second intermediate subset of imaginginformation. In another configuration, the apparatus may include imagecorrector/enhancer for generating a gamma corrected image byaccumulating gamma corrected subsets of imaging information. The gammacorrected image may have been subjected to both the gamma low contrastcurve and the gamma high contrast curve.

The aforementioned components of the disclosed system may be one or moreof the aforementioned components of the apparatus 1102 and/or theprocessing system 1214 of the apparatus 1102′ configured to perform thefunctions recited above. The processing system 1214 may include one ormore processor, switches, comparators, mathematical logic units, and/orcontrollers. For example, determiner for determining an imagingsensitivity value may include the processing system 1214 and/or memory1206. The image information comparator for comparing subsets of imaginginformation may include the processing system 1214. The imagecorrector/enhancer for applying a gamma correction may include theprocessing system 1214. The receiver for receiving linear imaginginformation may include a transceiver 1210 and/or a bus interface. Theconverter for converting the gamma corrected subset of imaginginformation may include the processing system 1214. The imagecorrector/enhancer for generating a gamma corrected image may includethe processing system 1214.

It is understood that the specific order or hierarchy of blocks in theprocesses/flowcharts disclosed is an illustration of exemplaryapproaches. Based upon design preferences, it is understood that thespecific order or hierarchy of blocks in the processes/flowcharts may berearranged. Further, some blocks may be combined or omitted. Theaccompanying method claims present elements of the various blocks in asample order, and are not meant to be limited to the specific order orhierarchy presented.

The previous description is provided to enable any person skilled in theart to practice the various aspects described herein. Variousmodifications to these aspects will be readily apparent to those skilledin the art, and the generic principles defined herein may be applied toother aspects. Thus, the claims are not intended to be limited to theaspects shown herein, but is to be accorded the full scope consistentwith the language claims, wherein reference to an element in thesingular is not intended to mean “one and only one” unless specificallyso stated, but rather “one or more.” The word “exemplary” is used hereinto mean “serving as an example, instance, or illustration.” Any aspectdescribed herein as “exemplary” is not necessarily to be construed aspreferred or advantageous over other aspects. Unless specifically statedotherwise, the term “some” refers to one or more. Combinations such as“at least one of A, B, or C,” “one or more of A, B, or C,” “at least oneof A, B, and C,” “one or more of A, B, and C,” and “A, B, C, or anycombination thereof” include any combination of A, B, and/or C, and mayinclude multiples of A, multiples of B, or multiples of C. Specifically,combinations such as “at least one of A, B, or C,” “one or more of A, B,or C,” “at least one of A, B, and C,” “one or more of A, B, and C,” and“A, B, C, or any combination thereof” may be A only, B only, C only, Aand B, A and C, B and C, or A and B and C, where any such combinationsmay contain one or more member or members of A, B, or C. All structuraland functional equivalents to the elements of the various aspectsdescribed throughout this disclosure that are known or later come to beknown to those of ordinary skill in the art are expressly incorporatedherein by reference and are intended to be encompassed by the claims.Moreover, nothing disclosed herein is intended to be dedicated to thepublic regardless of whether such disclosure is explicitly recited inthe claims. The words “module,” “mechanism,” “element,” “device,” andthe like may not be a substitute for the word “means.” As such, no claimelement is to be construed as a means plus function under 35 U.S.C. §112(f) unless the element is expressly recited using the phrase “meansfor.”

What is claimed:
 1. An image processor for adjusting luminosity of ahigh-dynamic range (HDR) image based on imaging sensitivity of a deviceconfigured to display the HDR image, the image processor comprising:electronic memory configured to store at least one gamma low contrastcurve and at least one a gamma high contrast curve; an imaging apparatusfor generating image data of a captured HDR image; an imagingsensitivity determiner configured to determine an imaging sensitivityvalue of a display device based on a input value of the display devicethat provides a predetermined output luminance at the display device; animage information comparator configured to compare the determinedimaging sensitivity value of the display device with a luminance valueof the generated image data; a gamma curve selector configured to selectthe at least one gamma low contrast curve when the luminance value isless than the determined imaging sensitivity value of the display deviceand select the at least one gamma high contrast curve when the luminancevalue is greater than the determined imaging sensitivity value of thedisplay device; an image corrector configured to correct the image dataof the captured HDR image by applying the selected at least one gammahigh contrast curve or the selected at least one gamma low contrastcurve to the generated image data to adjust the luminosity of the HDRimage; and an image display configured display the adjusted HDR image onthe display device, wherein the imaging sensitivity determiner isconfigured to determine the imaging sensitivity value of the displaydevice when the input value results in the predetermined outputluminance as a set breakpoint of candela per square meter (cd/m²) at thedisplay device.
 2. The image processing system of claim 1, wherein theset breakpoint is 100 candela per square meter (cd/m²) at the displaydevice.
 3. The image processing system of claim 1, further comprising animage data converter configured to convert the generated image data fromlinear imaging information to logarithmic imaging information, andwherein the luminance value is determined from the logarithmic imaginginformation.
 4. The image processor of claim 1, wherein the gamma curveselector is configured to compare the luminance value of the generatedimage data with a luminance value associated with the imagingsensitivity value of the display device.
 5. The image processor of claim1, wherein the image corrector is configured to correct the image databy normalizing the generated image data based on a normalization value,wherein the gamma curve selector is configured to select the at leastone gamma low contrast curve if the luminance value of the generatedimage data is less than a first luminance associated with the imagingsensitivity value, wherein the gamma curve selector is configured toselect the at least one gamma high contrast curve if the luminance valueof the generated image data is greater than a second luminanceassociated with the imaging sensitivity value greater than the firstluminance, and wherein the image corrector is configured to apply theselected gamma low contrast curve or the selected gamma high contrastcurve to the generated and normalized image data to obtain anintermediate subset of imaging information.
 6. The image processor ofclaim 5, wherein the normalization value is equal to the imagingsensitivity value of the display device.
 7. The image processor of claim5, wherein the image corrector is further configured to apply theselected gamma contrast curve by adding the normalization value to theintermediate subset of imaging information to obtain a gamma correctedsubset of imaging information.
 8. The image processor of claim 7,wherein the image corrector is further configured to convert the gammacorrected subset of imaging information from a logarithmic format to alinear format to generate the adjusted HDR image.
 9. The image processorof claim 5, wherein the image corrector is further configured to computea maximum total value by adding the intermediate subset of imaginginformation with a maximum differential gain value and compute a secondintermediate subset of imaging information by adding the normalizationvalue to the intermediate subset of imaging information, and wherein theimage corrector corrects the image data of the captured HDR image basedon a smaller of the computed maximum total value or the computed secondintermediate subset of imaging information.
 10. An image processor foradjusting luminosity of a high-dynamic range (HDR) image based onimaging sensitivity of a device configured to display the HDR image, theimage processing system comprising: electronic memory for storing aplurality of gamma contrast curves; an imaging apparatus for generatingimage data of a captured HDR image; an imaging sensitivity determinerconfigured to determine an imaging sensitivity value of a displaydevice; an image information comparator configured to compare thedetermined imaging sensitivity value of the display device with aluminance value of the generated image data to generate a gamma curveselection determination; an image data converter configured to convertthe generated image data from linear imaging information to logarithmicimaging information, with the luminance value being determined from thelogarithmic imaging information; a gamma curve selector configured toselect at least one gamma contrast curve from the plurality of gammacontrast curves based on the generated gamma curve selectiondetermination; and an image corrector configured to correct the imagedata of the captured HDR image by applying the selected at least onegamma contrast curve to the generated image data to adjust theluminosity of the HDR image.
 11. The image processor according to claim10, wherein the electronic memory stores at least one gamma low contrastcurve and at least one a gamma high contrast curve.
 12. The imageprocessor according to claim 11, wherein the gamma curve selector isfurther configured to select the at least one gamma low contrast curvewhen the luminance value is less than the determined imaging sensitivityvalue of the display device and to select the at least one gamma highcontrast curve when the luminance value is greater than the determinedimaging sensitivity value of the display device.
 13. The image processoraccording to claim 12, wherein the image corrector is further configuredto correct the image data of the captured HDR image by applying theselected at least one gamma high contrast curve or the selected at leastone gamma low contrast curve to the generated image data to adjust theluminosity of the HDR image.
 14. The image processor according to claim10, further comprising an image display configured display the adjustedHDR image on the display device.
 15. The image processor according toclaim 10, wherein the imaging sensitivity value is preconfigured withinthe display device.
 16. The image processor of claim 10, wherein theimaging sensitivity determiner is further configured to determine theimaging sensitivity value of the display device when an input valueresults in an output luminance of 100 candela per square meter (cd/m²)at the display device.
 17. The image processor processing system ofclaim 10, wherein the image corrector is configured to correct the imagedata by normalizing the generated image data based on a normalizationvalue, wherein the gamma curve selector is configured to select the atleast one gamma low contrast curve if the luminance value of thegenerated image data is less than a first luminance associated with theimaging sensitivity value, wherein the gamma curve selector isconfigured to select the at least one gamma high contrast curve if theluminance value of the generated image data is greater than a secondluminance associated with the imaging sensitivity value greater than thefirst luminance, and wherein the image corrector is configured to applythe selected gamma low contrast curve or the selected gamma highcontrast curve to the generated and normalized image data to obtain anintermediate subset of imaging information.
 18. The image processor ofclaim 17, wherein the image corrector is further configured to apply theselected gamma contrast curve by adding the normalization value to theintermediate subset of imaging information to obtain a gamma correctedsubset of imaging information.
 19. The image processor of claim 18,wherein the image corrector is further configured to convert the gammacorrected subset of imaging information from a logarithmic format to alinear format to generate the adjusted HDR image.
 20. The imageprocessor of claim 18, wherein the image enhancer is further configuredto convert the gamma corrected subset of imaging information from alogarithmic format to a linear format to generate the enhanced imageconfigured to be displayed on the display device.
 21. An image processorfor setting a luminosity of an image based on an imaging sensitivity ofa display device, the image processor comprising: a luminance adjustmentselector configured to select at least one luminance adjustment modifierby comparing a luminance of a captured image with an imaging sensitivityof a display device to determine an optimal luminance level for thecaptured image when displayed on the display device, wherein theluminance adjustment selector selects at least one gamma low contrastcurve when the luminance of the captured image is less than the imagingsensitivity and at least one gamma high contrast curve when theluminance of the captured image is greater than the imaging sensitivity;and an image enhancer configured to generate an enhanced image byapplying the selected at least one gamma high contrast curve or theselected at least one gamma low contrast curve to the captured image,such that the generated enhanced image is configured with the optimalluminance level when displayed on the display device; and an imagedisplay of the display device that is configured to display thegenerated enhanced image.
 22. The image processor of claim 21, whereinthe luminance adjustment selector is configured to select at least onegamma contrast curve from a plurality of gamma contrast curves based asthe at least one luminance adjustment modifier.
 23. The image processorof claim 22, further comprising an image information comparatorconfigured to compare the imaging sensitivity of the display device witha luminance of the captured image data to generate a gamma curveselection determination.
 24. The image processor of claim 23, whereinthe luminance adjustment selector is configured to select the at leastone gamma contrast curve based on the generated gamma curve selectiondetermination.
 25. The image processor according to claim 22, whereinthe luminance adjustment selector is further configured to select aluminance adjustment modifier to attenuate the luminance of the capturedimage when the luminance of the captured image is less than the imagingsensitivity of the display device and to select a luminance adjustmentmodifier to amplify the luminance of the captured image when theluminance of the captured image is greater than the imaging sensitivityof the display device.
 26. The image processing system according toclaim 21, wherein the imaging sensitivity of the display device ispreconfigured within the display device.
 27. The image processor ofclaim 21, wherein the imaging sensitivity of the display device isdetermined based on an input value that results in an output luminanceof 100 candela per square meter (cd/m²) at the display device.
 28. Theimage processor system of claim 21, wherein the image enhancer isconfigured to correct the image data by normalizing the generated imagedata based on a normalization value, wherein the luminance adjustmentselector is configured to select the at least one gamma low contrastcurve if the luminance of the generated image data is less than a firstluminance associated with the imaging sensitivity, wherein the luminanceadjustment selector is configured to select the at least one gamma highcontrast curve if the luminance of the generated image data is greaterthan a second luminance associated with the imaging sensitivity greaterthan the first luminance, and wherein the image enhancer is configuredto apply the selected gamma low contrast curve or the selected gammahigh contrast curve to the generated and normalized image data to obtainan intermediate subset of imaging information.
 29. The image processorof claim 28, wherein the image enhancer is further configured to applythe selected gamma contrast curve by adding the normalization value tothe intermediate subset of imaging information to obtain a gammacorrected subset of imaging information.